DOCUMENT RESUME 



ED 351 343 



TM 019 144 



AUTHOR 
TITLE 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



Gray, Patrick 

District-Level Perspective on Innovative Approaches 
in Performance Appraisal: Integration and 
Refinement • 
Apr 92 

14p,; Paper presented at the Annual Meeting of the 
American Educational Research Association (San 
Francisco, CA, April 20-24, 1992). 
Reports - Descriptive (141) — Speeches/Conference 
Papers (150) 



IDENTIFIERS 



MFOl/PCOl Plus P 
Educational Asse 
Educat i onal Impr 
^Elementary Scho 
Education; '^Eval 
Peer Evaluat ion; 
Administration; 
Teachers; Summat 
Teaching Methods 
'''Dade County Pub 
Evaluation 



ostage. 

ssment; Educational Cooperation; 
ovement ; Educat i onal Innovat i on; 
ol Teachers; Elementary Secondary 
uation Methods; Formative Evaluation; 

Research Utilization; School 
''^School Districts; ''^Secondary School 
ive Evaluation; ^Teacher Evaluation; 

lie Schools FL; ''Terf ormance Based 



ABSTRACT 

The Dade County (Florida) Public School System is 
replacing its Teacher Assessment and Development System (TADS), 
initiated in 1982, with systems that draw on recent research, link 
teaching skills to school improvement, and recognize and reward 
advanced pedagogy. The comprehensive approach will integrate 
subordinate and peer assessment through a dual focus on teaching and 
administrative contributions to school improvement. Independent of 
the TADS monitoring system, a Peer Intervention and Assessment 
Program (PIAP) is to be piloted in 1992-93. This formative and 
collaborative approach is, of necessity, linked to the summative 
realities of evaluation. The TADS was a ground-level instrument to 
a^ssess teacher performance in the classroom. The refinements that are 
proposed must: (1) assess performance above the minimum level; (2) 
relate performance behaviors to the teaching of thinking skills and 
the use of higher order questions; (3) use current research and data 
on effective practice; and (4) assesB advanced pedagogical 
techniques. The diversity of student clients and the contributions of 
educational professionals compel the use of multiple assessment 
systems. School improvement must be the forum within which 
instructional and administrative contributions are measured. (SLD) 
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im-RODUCTIOH 

Since the implementation in 1982 of the Teacher Assessment 
and Development System (TADS), the Dade County Public Schools has 
mirrored the nationally evolving experience with the refinement of 
performance appraisal. Over the course of ten years, some half 
million performance observations have enabled a focus on 
measurable school improvement and a concurrent move toward 
participatory management. 

Alignment of performance appraisal and school improvement in 
an atmosphere of participatory management, while not an identified 
target objective of the implementation of TADS, has nonetheless 
been a major effect. Drawing from this experience, the Dade 
County Public Schools has initiated actions to replace TADS with 
systems that draw from recent research, link individual teaching 
skills to school improvement plans, and recognize and reward 
advanced pedagogy. 

A comprehensive approach will integrate subordinate and peer 
assessment through a dual focus on both teaching and 
administrative contributions to school improvement. These 
contributions represent a broad expansion of the required teaching 
behaviors on which TADS is based, and center more on professional 
responsibilities and those teaching behaviors (both required and 
auxiliary) within which creative thinking is recognized. 
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HISTORY 

The formative nature of performance assessi^^nt was 
exemplified by the management/labor collaboration in the 
development of TADS and its pilot implementation in 1982* 
Throughout the research and development phase, the joint focus was 
on the professionalism of the instructional staff. In both the 
1982-83 pilot and the 1983-84 field test years the 
labor/management working harmony emphasized the joint commitment 
to the formative dimension of performance assessment. 

All orientation sessions in the pilot and field test years 
were jointly presented. Joint presentations continue to be 
provided at the twice-annual orientation for new teachers. 
Refinements in the initial system, as identified through the pilot 
and field test experiences, were jointly presented. 

Training was provided for both administrators and teacher 
union stewards. The exactness of the training supports the joint 
commitment. 

PROFICIENCY AND MONITORING 

Proficiency updates, expressly designed to prevent "coder 
drift," were never implemented, primarily due to budget 
constraints. Instead, special proficiency updates concentrated on 
specific dimensions of TADS which were shown to require ongoing 
emphasis, interpretation, and training. These included such areas 
as lesson closure, identification of specifics of observed 
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deficiencies, timeliness of prescriptive assistance, the role of 
homework. 

The joint commitment for orientation and training was 
noticeably absent from the proficiency updates, also for budgetary 
reasons rather than design. While this was not indicative of a 
lessening of the joint commitment, it had the same effect. 
Several major problems arose as a result of this practice: 

1. inconsistency arose in the post-training 
interpretations; 

2. significant increases occurred in disputed practices; 

3. "coder drift" increased. 

The fix for this experience was the use of a program 
monitoring model implemented immediately after the first full year 
of district-wide experience with TADS. 

The TADS Program Monitoring Model consisted of four basic 
components: 

1. role specifications of key personnel 

2. monitoring activities of labor and management 

3. implementation reviews 

4*. timelines for monitoring activities. 

Consistency in interpretations and applications was 
addressed, in part, through joint observations by principals and 
assistant principals of lessons by volunteer teachers, enabling 
comparison of observation data and discussion of rationales for 
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ratings of acceptable and unacceptable teaching performances. 
(Data were not aggregated into formal assessment records for 
summative purposes). 

In conjunction with the other monitoring components, the 
consistency checks should have indicated elements of coder drift. 
It is likely that this would have occurred if the program 
monitoring model had been retained; however, the sheer involvement 
in time and effort contributed to an alternative process which 
replaced the highly structured TADS Program Monitoring Model: the 
TADS Monitoring Committee. 

By design, rather than concentrating on the original 
monitoring components, the TADS Monitoring Committee serves "as a 
forum for attempting resolution of observation/evaluation issues 
prior to the filing of a grievance." 

Rather than an alternate means of assuring consistency in 
the implementation of TADS, the TADS Monitoring Committee was 
actually part of an alternative to the formal grievance process at 
two of the three levels before binding arbitration. While it 
formally retained the authority to conduct visits, (an action that 
was part of the original monitoring process), even the site visits 
were defined as "for the purpose of resolving disputes and 
monitoring and enhancing TADS implementation." 

In 1987-88, the fourth year of full district implementation 
of TADS, the highly structured process of monitoring consistency 



ERLC 



6 



s 



gave rise to an even greater structure of dispute resolution. The 
hope was for identification of system-wide issues of consistency 
through joint resolution of disputes. This hope was flawed in its 
very concept, and contributed to the very inconsistency it was 
designed, in part, to address. 
PEER ASSESSMENT 

To the extent that involvement of individuals in setting the 
standards for their own performance is one touchstone of 
professionalism, focus on the professional ization of teaching 
generated, independent of the TADS monitoring process, a 
negotiated agreement to pilot a peer intervention and assistance 
program in the Dade County Public Schools in 1992-93, subject to 
availability of funding. 

Many of the components of the Peer Intervention and 
Assessment Program (PIAP) were drawn from at least two years of 
experience at selected schools having implemented school based 
management/shared decision making programs, within which 
variations of peer assessment were operational. A joint 
labor/management committee organized to oversee those initial 
experiences with peer assessment met only once and generated no 
formal reports. That committee did not meet with, share data 
with, or interact with the TADS Monitoring Committee. 

A joint labor/management committee was established for the 
express purpose of assuring orderly and equitable implementation 
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of PIAP, and recommending any additional guidelines and 
definitions as may be necessary. The only linkage with TADS was 
in the mandatory TADS training for consulting teachers; this 
supplemented training in a variety of peer intervention and 
assistance strategies. 
FPRHATIVE GROWTH 

Implementation of peer observation models through shared 
decision-making and school -based management serves to illustrate 
the formative and collaborative dimensions, even where these 
dimensions have evolved in parallel. The extensive experience 
over a ten-year period has shown that the most effective 
instructional assistance provided to raise competencies to 
acceptable standards is quality time with a master teacher. Both 
assistance reviews and external reviews corroborate the teaching 
skills needed, while the attainment of those skills most 
frequently occurs through the direct assistance of a master 
teacher. 

In Florida, a teaching certificate is initially earned 
through satisfaction of requirements of a Professional Orientation 
Program, for which the peer teacher and the professional 
development plan are intrinsic to the growth and development of 
new and lesser experienced teachers, and to veteran teachers who 
need to strengthen certain skills. 
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Personal orientation to all criteria and procedures of the 
assessment system allows for that valuable focus on instructional 
skills and expectations. This focus is particularly critical to 
principals, as instructional leaders, as time constraints and non- 
teaching issues otherwise divert their attention away from the 
primary teaching and learning mission. Teachers know, immediately 
and conclusively, the shared commitment to quality instruction. 
SlIMHATIVE REALITY 

No matter how formative, in concept and practice, assessment 
is linked to summative realities. Employment standards, while 
expressed in assessment criteria, are implemented through such 
realities as licensure, employment and reappointment, contract 
status, incentives and rewards (both programmatic and financial), 
assignments and promotions. These realities diminish the 
commitment to the formative/growth dimensions of any assessment 
system. One measure of the commitment to the formative/growth 
dimension is the degree of structure of the dispute resolution 
process related to assessment of performance. 

Another measure of shared commitment to the formative 
aspects of TADS is the linkage with professionalization 
initiatives, including school-based management/shared decision- 
making programs, peer intervention and assistance programs, 
faculty council/shared decision-making cadres, and related joint 
collective bargaining professionalization issues. 
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FVTURE DEVELOPMENT 

Included among the refinements gleaned from ten years of 
TADS experience are the need for automating the system, avoidance 
of unnecessary instruments, strengthening of proficiency 
monitoring nd attendant training, and focus on advanced pedagogy 
through incorporation of critical thinking dimensions, 

TADS was constructed to assess '•acceptable" teacher 
performance in the classroom; as such, it is considered a ground- 
level instrument. The spirit of professionalism makes assessment 
of merely acceptable performance at a base level unreasonable. 

The TADS research base includes effective teaching research 
from the mid 1970* s through 198L Since that time effectiveness 
teaching research has made great strides in several areas which 
will be included in future assessment systems which measure more 
than adequate performance. This research addresses, in addition 
to critical thinking skills, questioning techniques, (including 
higher order questions), development of homework assignments to 
reinforce and promote instruction, expert teacher techniques, 
motivational activities, demonstration of warmth and friendliness, 
and cooperative learning. None of these is part of TADS. 

Advanced pedagogical techniques are required for approval of 
an evaluation system by the state of Florida. Auxiliary teaching 
behaviors, either in the aggregate or in combination with required 
teaching behaviors, do not meet this requisite. 
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Refinements to TADS, therefore, must carry out the 
following: 

1. assess performance above the minimum level of 
acceptabil ity; 

2. relate performance behaviors to the teaching of 

' thinking skills and the use of higher order questions; 

3. use current research and data on effective practices; 

4. assess advanced pedagogical techniques. 

In the state of Florida, assessment systems must also have a 
direct linkage with school improvement plans. 

Support for thase refinements is found in "Assessment of 
Educational Personnel in the Twenty-First Century," in which 
Tetenbaum and Mulkeen (1988) stress the need for an evaluation 
system to assess the primary goal of education in the twenty-first 
century: to educate students who can use a variety of thinking 
strategies for solving problems and exploring new ideas. 

In "A New Generation of Classroom-Based Assessments of 
Teaching and Learning; Concepts, Issues and Controversies from 
Pilots of the Louisiana Star," Ellett and Garland (1987) report 
that the second generation of teacher assessment must focus on 
student learning, and that teacher performance is credited only if 
the teaching behavior has an effect on the learner. They 
emphasize the need for extensive teacher planning to include 
assessment materials that adequately reflect student attainment of 
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objectives, and stratified home learning assignments to enhance 
learning. Motivation for learning and providing students with 
the reason for a task are elements missing in TADS that are 
t Mtical to the psychosocial learning environment. 

Under a broader heading of "Professional Responsibilities," 
the contributions a teacher makes to a school's improvement plan 
are likely to be measured by criteria far beyond effective 
teaching behaviors identified in a classroom setting. In fact, 
some counties in Florida have begun piloting a usage of the more 
standard assessment systems fo"* novice and annual contract 
teachers, with tenured/long-term teachers eligible to opt for a 
mutual identification of professional responsibilities 
contributions to the school improvement plans as a basis for 
performance assessment. This extends performance well beyond the 
classroom setting and recognizes and rewards contributions 
typically not measured. The diversity and variability of such 
contributions moves away from any "one size fits all" system, and 
compels a process which empowers individual improvement plans, 
SUMMARY 

Second generation assessment in the Dade County Public 
Schools is highly focused on process, intrinsically research- 
based, linked to school improvement, and inclusive of the 
relationship of both instructional and administrative skills. 
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The process of development continues to be a mutual 
involvement of both management and labor. Practitioner input 
draws from those standards identified through peer assessment and 
participatory management. The "government in the sunshine" 
mandate in Florida includes parents, business partners with 
education, and school advisory groups in the expanded experience 
of participatory management. 

Research continues to be fundamental to assessment, but it 
has significantly exceeded the focus on classroom competencies, 
and includes such standards as The Personnel Evaluation Standards 
from The Joint Committee on Standards for Educational Evaluation. 
This enables a focus on the competency of the evaluation process 
itself relative to propriety, utility, feasibility, and accuracy. 
The availability of such research, in conjunction with the ten- 
year experience with TADS, broadens the second generation focus. 
School improvement, to the extent that it represents ' ^ best of 
individual competencies in conjunction with learning outcomes, 
must be the forum within which the confluence of both 
instructional and administrative contributions must be measured. 
That measurement profoundly involves the very parents, neighbors 
and citizens directly linked to the individual school. 

The diversity of the student clients and the contributing 
educational professionals compels the utilization of multiple 
assessment systems not constricted by the convenience of service 
characteristics. 
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